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This study aimed to conduct a psychometric evaluation of the Indonesian 
version of the Communities Advancing Resilience Toolkit Assessment 
Survey (CART-AS) instrument with validity and reliability testing. A cross- 
sectional study was conducted among 280 respondents. The translation was 
done using the Beaton method. The psychometric test process was 
conducted by testing the construct validity and reliability. Confirmatory 
factor analysis (CFA) values indicated that the model fits the previous model 


Keywords: with the Chi-square/df=1.37 conformity indicator; comparative fit index 
Community (CFD=0.99; goodness of fit index (GFI)=0.90; root mean square of 
; approximation (RMSEA)=0.03; and standardized root mean square residual 
Cultural adaptation (SRMR)=0.03. In the CFA measurement, the convergent validity value or 
Reliability the loading factor value of each indicator was also found in the range of 
Resilience 0.55-0.99. The average variance extracted (AVE) values on the five 
Validity constructs of the CART-AS instrument show values in the range of 0.55- 
0.98. The results of the reliability test showed the Cronbach alpha value of 
0.94. The Indonesian version of the CART-AS instrument is a valid and 
reliable instrument to measure the resilience of disaster-affected 

communities in Indonesian culture. 
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1. INTRODUCTION 

One of the crises and disasters currently happening is the COVID-19 pandemic. According to the 
COVID-19 dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins 
University (JHU), this outbreak later transformed into a pandemic, and has become a global health and 
economic disaster on an unprecedented scale. As of March 15, 2022 more than 460 million people worldwide 
have been infected with COVID-19, resulting in over six million deaths and the incidence of COVID-19 in 
Indonesia is the highest in Southeast Asia. In Indonesia, according to a report by the Indonesian COVID-19 
Task Force, this global disaster has become a serious public health concern and has been designated a 
national disaster. As of March 15, 2022 more than 5.9 million Indonesians have been infected with as many 
as 152,437 deaths [1], [2] 
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In Indonesia, various regulations and institutions related to disaster risk reduction (DRR) have made 
significant progress, with the enactment of the disaster management law no. National disaster management 
number | of 2019. The National Disaster Management Agency (Badan Nasional Penanggulangan Bencana/ 
BNPB) was established with a mandate to improve coordination of DRR responsibilities between 
government agencies, non-governmental organizations, international partners, and other stakeholders. 
Indonesia has developed internationally recognized good practices in community-based emergency response 
and post-disaster recovery. However, challenges remain including coordination, human resources and 
technical capacity, systematic consideration of risks in development, infrastructure resilience, and the 
establishment of sustainable and efficient financing mechanisms related to DRR and disaster preparedness 
and recovery [3], [4]. 

Resilience can help improve responses to disaster risk because it requires holistic consideration of 
hazards, exposure, risks, vulnerability, and capacity. Programs undertaken to improve disaster resilience can 
save lives while protecting infrastructure, livelihoods, social systems, and the environment. Resilience as a 
multidimensional concept has been known to have the potential to contribute to identifying effective and 
efficient options to reduce and manage current and future risks [5], [6]. 

A study conducted in Yogyakarta, it is known that community resilience to disasters in Sleman 
Regency, especially Cangkringan District, is generally still low [7]. There are areas that have very low 
resilience, but other areas show a high level of resilience [8]. Measurement of community resilience to 
disasters has a high urgency, including identifying priority needs for continuous improvement, monitoring 
progress in the community, and comparing the success of efforts to increase resilience with the costs that 
have been incurred. Establishing baselines or reference points for measuring changes in resilience over time 
is essential. If the measurement of resilience is not done properly and validly, then the efforts that are made 
to increase community resilience and preparedness to disasters will be disorganized and have no clear 
improvement goals [9], [10]. 

Recognizing the importance of community resilience in disaster management policies and practices 
has resulted in the need for instruments to assess resilience at the community level [11]. From the results of 
the literature study conducted on the instruments for evaluating community resilience to disasters, it was 
found that several instruments were community-based and could be used because they were questionnaire- 
based with a quantitative analysis approach. The instruments are the Community Disaster Resilience 
Scorecard compiled by the Torrens Resilience Institute Toolkit and the Conjoint Community Resilience 
Assessment Measure compiled by Leykin et al. in 2013 [12], [13]. The instruments mentioned are more 
focused on measuring the current state of the community, without developing disaster management plans in 
the future for increasing community resilience after the measurement. Good disaster management planning is 
one of the most important steps towards future community recovery after a disaster [14]. 

The Communities Advancing Resilience Toolkit Assessment Survey (CART-AS) instrument is 
known as an instrument that has elements of disaster management in assessing community resilience. CART 
is an integrated system with specific interventions that are made in order to increase community resilience 
through assessment, planning, and action. CART directly involves community organizations in developing 
and implementing strategies to build community resilience. CART can help civil society organizations 
(CSOs) to assess their communities systematically in disaster management, including pre-disaster, during 
disaster and post-disaster [11]. In a study conducted by Lee ef al. in 2018 in Mississippi, CART-AS was 
psychometrically tested with Cronbach's alpha values of 0.86 for connection and caring, 0.84 for resources, 
0.90 for transformative potential, and 0.85 for disaster management [15]. 

Resilience is defined as the ability or capacity, which is related to successful adaptation, for 
recovery from adversity. Building a resilient community requires more than bringing individuals together as 
a group. Community resilience requires the participation of the entire community to effectively cope with, 
and learn from, adversity. A resilient community has the ability to change the environment through deliberate 
collective action. CART is known to provide information on the current level of community resilience and 
use that information to build better community resilience. 

Based on the information, the researchers aimed to determine the results of the CART-AS 
psychometric evaluation, especially in communities affected by the COVID-19 pandemic in the Special Region 
of Yogyakarta (DIY), Indonesian. Sleman Regency was chosen because it has the highest COVID-19 incidence 
rate in the DIY Province. This instrument has never been adapted into the Indonesian language and culture. The 
researchers aimed to test the validity and reliability of the Indonesian version of the CART-AS instrument. The 
researchers used the Beaton's approach to adapt the CART-AS instrument to the culture of the Indonesian 
people. 


2. RESEARCH METHOD 
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Translation and psychometric evaluation of the Communities Advancing Resilience Toolkit 
Assessment Survey (CART-AS) instrument were performed. CART-AS is an instrument that consists of 27 
question items in five domains, including connection and caring, resources, transformative potential, 
information and communication and disaster management. There are eight questions on the connection and 
caring, four questions on resources, four questions on transformative potential, six questions on the 
information and communication, and five questions on the disaster management domain. Each question uses 
a Likert-scale with a score range of 1-5, with a higher score indicating better disaster resilience. The 
translation and cultural adaptation process was conducted systematically by applying the Beaton method 
which consists of five stages, including forward translation, synthesis of the translations, back translations, 
expert committee, and test of the prefinal version [16]. The ethical considerations of this study have received 
approval from the Medical and Health Research Ethics Committee (MHREC) Faculty of Medicine, Public 
Health and Nursing, Universitas Gadjah Mada with number KE-FK-0548-EC-2021. 

In the forward translation stage, two native Indonesian speakers fluent in English were involved. 
This translation process resulted in two Indonesian translated versions of the instrument, which were coded 
T1 and T2. After producing the T1 and T2 instruments, the authors synthesized and combined the two 
instruments, and the results of this combination were given the code T-12. The results of the synthesis (T-12) 
were then translated back into the original language of the instrument. The translators who did the back 
translation were two English native speakers, and the translation results were coded BT1 and BT2. In the 
expert committee review, there were two experts involved, including one expert on the context of the 
instrument (disaster management) and one expert on methodology in instrument development. After BT1 and 
BT2 were reviewed, the authors pre-tested the instrument to respondents. 

The process of psychometric evaluation included validity and reliability tests. The sample in this 
study was people who live with permanent housing in the village of Purwobinangun, Pakem, Sleman, 
Indonesia. The number of samples used in this study were 315 people, with details of 35 respondents 
involved in the pre-testing stage, while 280 respondents participate in testing the validity and reliability. Data 
collection was conducted in the period June-July 2021 in the working area of the Kalasan Health Center, 
Kalasan, Sleman, Indonesia. Construct validity testing was conducted using confirmatory factor analysis 
(CFA), and reliability testing was conducted using Cronbach’s alpha coefficient. The CFA test was 
conducted by evaluating the root mean square error approach (root mean square of approximation 
(RMSEA)<0.08), goodness-of-fit (GFI>0.9), Tucker-Lewis index (TLI>0.9), comparative suitability index 
(CFI>0.95) and normed conformity index (NFI>0.95). The internal consistency was measured using 
Cronbach’s alpha coefficient, comparing each item with all other items. A minimum score of 0.70 was set to 
ensure adequate reliability. The authors also calculated the construct validity by determining the average 
value of variance extracted (AVE) between items or indicators. AVE is calculated as the total square of the 
standardized factor loading divided by the total square of the standardized factor loading plus the total 
variance of error. The AVE value of 0.50 indicates a good convergent. 


3. RESULTS AND DISCUSSION 
3.1. The description of respondent’s characteristic 

Based on primary data, the number of female respondents is almost three times higher than the 
number of male respondents. The age of the respondents was predominately in the age range of 18-35 years 
(63.9%). In the latest education data, it is found that most respondents have the last education level of high 
school (89.3). Characteristics of respondents can be seen in Table 1. 


Table 1. Characteristic of respondent (n=280) 


Respondent’s characteristics Frequencies (f) Percentage (%) 
Gender 

Female 205 73.2 
Male 75 26.8 
Age (year) 

18-35 179 63.9 
36-55 91 32.5 
>55 10 3.6 
Education 

Elementary 4 1.4 
Junior high school 3 1.1 
Senior high school 250 89.3 
Bachelor’s degree 18 6.4 
Master’s degree 5 1.8 
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3.2. The translation of CART-AS 

In the process of cultural adaptation using the Beaton method, the stages that were done included 
discussing with experts about the appropriateness of the meaning and redaction of the translations by the 
researchers. The selection of these experts is based on the World Health Organization (WHO) guidelines, 
involving experienced experts in the field of instrument development and in the context of the instrument. 
The experts involved are methodological experts who have previously developed/translated instruments and 
experts in accordance with the context of the questionnaire, namely disaster risk reduction (DRR)/disaster 
science experts. 

In the expert panel discussion process, some feedback and suggestions for editorial changes were 
obtained from the experts. The input from the experts was immediately followed-up during the discussion 
until it was agreed that the editing was the best and all the experts agreed. There were three minor revisions 
to the original instrument version which were finally approved by all experts. Items that have been revised 
are numbers 13, 15 and 23. 


3.3. The validity test of CART-AS 

The measurement of construct validity in this study was done by using CFA. Testing the Kaiser- 
Meyer-Olkin (KMO) value and Bartlett’s test were first done to see whether the conditions were met or not 
before doing factor analysis. The KMO value of sampling adequacy is 0.898 with p<0.01 and Bartlett's Test 
of Sphericity is 2=13423.078 indicating an acceptable value. The measurement sample adequacy (MSA) 
value is in the range of 0.809-0.945. 

Based on the next factor extraction, the value of the principle component analysis (PCA) was 
determined with varimax rotation. The extraction results identified five factors that have an eigenvalue >1, 
namely factor one with an eigenvalue of 11.078, factor two with an eigenvalue of 4.496, factor three with an 
eigenvalue of 2.647, factor four with an eigenvalue of 2.038 and factor five with an eigenvalue of 1.400. The 
five factors formed were able to explain the total cumulative variance of 80.221%. The five factors formed 
are the same as the results of previous studies [17]. 

The CFA analysis was conducted to reconfirm whether the model formed was in accordance with 
the model predicted in previous studies. The indicators measured are the chi-square value divided by df, CFI, 
GFI, RMSEA, and standardized root mean square residual (SRMR). The results of the measurements are 
listed in Table 2. 


Table 2. CFA indicator measurement results (n=280) 


Indicator Value Standard Interpretation 
Chi-square/df 1.37 1-3 Accepted 
Comparative fit index (CFI) 0.99 >0.9 Accepted 
Goodness of fit index (GFI) 0.904 >0.9 Accepted 
Root mean square error of approximation (RMSEA) 0.037 <0.06 Accepted 
Standardized root mean square residual (SRMR) 0.034 <0.09 Accepted 


The results of the CFA analysis in this study also show a regression from the indicator variable to 
the latent variable. The calculated value is convergent validity or the loading factor value of each indicator. 
The loading factor value is accepted if it has an estimated value of 0.50. The estimation results are shown in 
Table 3. 

By using the convergent validity criterion of 0.50, all loading factors have good and acceptable 
values. All items show a good correlation and it can be concluded that community resilience can be explained 
by five factors DM, TP, IC, R and CC. In the CFA calculation, there is an adjustment to the modification 
indices method suggested by the data processing software. This adjustment results in a new covariance 
between items, as shown in Figure 1. 

In addition, the AVE value must be calculated for all latent constructs, namely five constructs 
including DM, TP, IC, R and CC. The results of calculating the AVE value in this study are listed in Table 4. 
Table 4 shows that all constructs have a value>0.5 which means they have a good convergent value. The 
results of construct validity in this study provide confidence that the indicator size taken from the sample can 
describe the real score in the population. 


Table 3. The value of the loading factor for each item 
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Loading Interpretation 


Factor 
CC8 0.58 Accepted 
CC7 0.56 Accepted 
CC6 0.99 Accepted 
CCS 0.55 Accepted 
CC4 0.58 Accepted 
CC3 0.55 Accepted 
CC2 0.99 Accepted 
CC1 0.91 Accepted 
R4 0.99 Accepted 
R3 0.99 Accepted 
R2 0.98 Accepted 
RI 0.99 Accepted 
IC4 0.97 Accepted 
IC3 0.99 Accepted 
IC2 0.97 Accepted 
IC1 0.98 Accepted 
TP6 0.56 Accepted 
TP5 0.75 Accepted 
TP4 0.74 Accepted 
TP3 0.78 Accepted 
TP2 0.86 Accepted 
TP1 0.83 Accepted 
DM5 0.98 Accepted 
DM4 0.73 Accepted 
DM3 0.98 Accepted 
DM2 0.98 Accepted 
DM1 0.99 Accepted 


DM= Disaster management; TP= Transformative potential, IC= Information and Communication; 
R= Resources; CC= Connection and Caring 


CMIN = 418.553 
DF = 304 


CFI = .992 
GFI = .904 
RMSEA = .037 
SRMR = .034 


Figure 1. CART-AS CFA model diagram 
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Table 4. The average of variance extracted (AVE) for each construct 
AVE Value Interpretation 


CC 0.55 Accepted 
R 0.98 Accepted 
IC 0.96 Accepted 
TP 0.58 Accepted 
DM 0.87 Accepted 


DM= Disaster management; TP= Transformative potential; IC= Information and Communication; R= Resources; 
CC= Connection and Caring 


3.4. The reliability test of CART-AS 

The last analysis, namely the reliability test is done by assessing the Cronbach alpha coefficient. If 
the Cronbach alpha value is closer to 1, the better the reliability of an instrument [18]. There are several 
categories of Cronbach alpha values, including: very low (<0.30), low (0.30 a 0.60), moderate (0.60 0.75), high 
(0.75<0.90), and very high (>0.90). The results of the reliability test of this study showed the Cronbach alpha 
value of 0.941. The reliability value of the instrument in this study is included in the category of high reliability. 


3.5. Discussion 

Based on the results of this study, the Indonesian version of the CART-AS instrument has good 
validity and reliability values. The factor analysis showed similar results to the theory and predictions of 
previous studies. There are no items omitted from the translation process to the reliability assessment. The 
total instrument items tested in this study were 27 items in five domains, including eight connection and care 
domain items, four resource domain items, four information and communication domain items, six 
transformative potential domain items and five disaster/crisis management domain items. The construct that 
composes each item in the Indonesian version of CART-AS is also the same as the original instrument 
construct [17]. 

The results of the cultural adaptation process with the Beaton method indicated there are editorial 
adjustments to several items of this CART-AS instrument. At the expert panel stage, there were three items 
that underwent minor adjustments in terms of the editorial used. The items that underwent adjustments were 
items numbered 13, 15 and 23. Editorial changes were made because according to experts, there was some 
terminology that were not understandable by ordinary Indonesian people and could lead to multiple 
meanings. In addition, the researchers have conducted initial experiments (field testing) on the instrument 
items and collected feedback from targeted respondents. Most respondents at this early stage said that the 
language used was easy to understand. The researchers also discussed the results with the original developer 
of the CART-AS instrument, and compared the back translation to the original language of the instrument 
and concluded that there was no significant change in meaning in the Indonesian version of CART-AS. 

The next process was testing the construct validity of the instrument. The measurement of the value 
of the construct validity of this instrument was conducted by using CFA statistical tests. In the CFA 
assessment, it is known that the suitability indicator Chi-square/df=1.37; comparative fit index (CFI)=>0.90 
(0.99); goodness of fit index (GFI)=0.90 (0.90); root mean square of approximation (RMSEA)=<0.06 (0.03); 
and standardized root mean square residual (SRMR)=<0.09 (0.03), so that no items were deleted. In addition, 
in the CFA calculation, there was an adjustment with the modification indices method. This adjustment 
resulted in the emergence of a new covariance between items. Modification Indices resulted in several 
recommendations for adding a dash that could reduce the chi-square value so that the model could better fit 
[19], [20]. 

In addition to looking at the loading factor of each item, CFA is also used to see the suitability of the 
formed domain with that predicted by the original instrument research. The results of the test show that the 
grouped items match and also they exactly match the model on the original instrument. The results of the 
CFA in this study are similar to the results of the original instrument development, namely the 
RMSEA=0.039, CFI=0.99. This study is also in accordance with the results of a previous study in China 
which aimed to evaluate the reliability and validity of the Chinese version of the Communities Advancing 
Resilience Toolkit (CART) which showed that the overall Cronbach alpha value of the CART-AS Chinese 
version was 0.96 and the Cronbach alpha for all dimensions of CART- The US Chinese version is between 
0.888-0.952. The CFA results show that for the CART-AS Chinese version, it has a GFI >0.9 (0.94), the root 
mean square error of approximation (RMSEA) <0.05 (0.04) which indicates the conformity of the construct 
validity of the CART-AS Chinese acceptable version [21]. 

The last aspect that was assessed in this study was the reliability of the instrument. This study uses 
the Cronbach alpha statistical test as a test tool. The reliability value of the Indonesian version of the 
CART-AS instrument in this study, overall, on 27 items was 0.941. A good reliability value indicates a 
strong correlation between instrument items. This reliability value is included in the high category, because it 
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is close to the number 1 [18], [22], [23]. The results of a good reliability test can be interpreted as the stability 
of the answers given by the respondents to the questions given in an instrument. Thus, the items in the 
Indonesian version of CART-AS can be trusted and consistently describe each domain contained in the 
instrument. The value of the reliability test in this study is in accordance with the original instrument research 
by Pfefferbaum et al. with a range of Cronbach alpha values in each domain of 0.59-0.70 which means it has 
a moderate category of reliability. This study is also in accordance with the results of a previous study in 
China by Man et al. in 2017, which shows the overall Cronbach alpha value of the CART-AS Chinese 
version is 0.966 and the Cronbach alpha for all dimensions of the CART-AS Chinese version is between 
0.888-0.952 which indicates good internal consistency of the CART-AS Chinese version [21]. 

The first dimension, namely connection and caring, includes connectedness, shared values, 
participation, social support and equality. Members of the community/CSO who feel that they need each 
other have a connection and concern with other members who need help in dealing with their life problems. 
This is in accordance with the important factors presented by Dreyer in 2015, namely protective factors 
which are important components in individual resilience that protect individuals from psychological disorders 
when depressed. Community connectedness also allows individuals to feel part of their community and 
makes them feel safe, where their needs can be met together with the community. This can make individuals 
resilient and can improve and maintain their psychological well-being [24]-[26]. 

The next dimension is resources, including natural, physical, social, and financial resources. This is 
also in line with the concept presented by Ungar in 2011, that individual resilience involves the community's 
ability to provide resources to meet their basic needs. Individual resilience can also be achieved by ensuring the 
ability to distribute resources properly [27]. To maintain and increase community resilience, 
Chandra et al. in 2013 underlined that communities must build capacities characterized by resilience to 
withstand stress, resilience to diversity, and resilience to resource mobilization speed. Therefore, this dimension 
is very relevant to the target community in this study, namely the Indonesian people in general [28]. 

The third dimension, namely transformative potential, includes the community's ability to interpret 
shared experiences, evaluate shared successes and failures, and evaluate negative experiences with critical 
analysis, so that they can set goals and develop strategies that can strengthen individual resilience in a 
community. With this transformation potential, the community could recognize risks, identify problems, and 
develop resolution strategies if problems occur in the form of solution options. Individuals in the community 
are expected to gain similar competencies and learning from their daily interactions. This competence will 
make individuals ready to face tough situations that occur and make them resilient [29]. 

The fourth dimension, namely disaster/crisis management, includes the community's ability to 
prevent, mitigate, prepare, and recover from disaster situations that occur. In addition, this dimension also 
describes the activities done by the community to avoid or control the crisis and prevent its impact from 
getting worse. The existence of this disaster management capability is a form of community resilience to 
disasters. Therefore, with disaster management competence in the community, individuals have the potential 
to be resilient in dealing with disasters or crises that can occur at any time [30]. 

The last dimension, namely information and communication, describes how accurate information 
can be communicated efficiently. Relevant and accurate information is an important component in the event 
of a disaster or crisis because communication allows community members to express their needs. If a 
communication culture is well-established in the community, they are more likely to respond to disasters with 
resilience. 


4. CONCLUSION 

The results of the psychometric test of the Indonesian version of the CART-AS instrument have 
been proven to be a valid and reliable instrument. The cultural adaptation process shows that all instrument 
items translated, have been recognized and agreed upon as good translations by the entire panel of experts 
including linguists, methodologists, and disaster context experts. In addition, in the discussion process with 
the original developer of the instrument, it was stated that the contents of the translated instrument were in 
accordance with the original meaning of the instrument. The construct validity test obtained the CFA results 
showed a good fit on the 27 items of the Indonesian version of the CART-AS instrument. The overall 
reliability value of the Indonesian version of the CART-AS instrument is 0.94. It is hoped that this instrument 
can be used in assessing community resilience in dealing with natural and non-natural disasters, so that it can 
be used as an indicator in preparing further interventions. 

In general, the translation results and the psychometric aspects of this instrument have met the 
requirements to be adapted/used in the Indonesian setting. However, the researchers still recommend future 
researchers who want to use this instrument to adjust it in several aspects, including the context of disaster 
events and local culture. 
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Appendix 1. Back-translation of community advancing resilience toolkit assessment 


survey Indonesian version 


Connection and caring domain 


OADM WNE 


People in my community feel they are part of the community. 

People in my community take responsibility for welfare of the community. 
People in my community have hopes for the future. 

People in my community help each other. 

My community treats others fairly and without caring about their backgrounds. 
My community supports programs designated for children and families. 
People in my community work together to foster the community. 

People in my community can be trusted. 

Resources domain 


9 My community has the resources it needs to handle issues it may face (for example, finances, information, technology, 
equipment, materials and services). 

10 People in my community can obtain the services they require. 

11 People in my community know where they need to go to get various tasks accomplished 

12 My community works together with organizations and agencies external to the community to accomplish various tasks. 
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Information and communication domain 


13 
14 
15 


16 


My community always provides information to others on important issues for them. 

Internal information on issues in my community is generally accurate and fair. 

The source of communication used by my community is effective in disseminating information to residents (for example, 
television, radio, newspaper, internet, telephone or local news agency). 

Communication and information in my community focuses on both positive and negative issues. 


Transformative potential domain 


17 
18 


22 


My community has effective leaders. 

People in my community establish communication with leaders who can help foster our community. 

My community reflect on both their successes and failures so as to learn from their experiences. 

My community develops their abilities and finds resources to overcome problems and achieve the goals they set for 
themselves. 

My community has priority and establishes goals for the future. 

People in my community trust the local officials/leaders. 


Disaster management domain 


23 
24 
25 
26 
27 


My community makes efforts to prevent disasters and crises. 

My community actively prepares itself for any future disasters and crises. 

My community is able to provide emergency services during a disaster or crisis. 
My community has a program and service to assist people post disaster or crisis. 
In the event of disaster or crisis, my community details what needs to be done. 
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